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(54) Title: SINGLE-CHIP ARCHITECTURE FOR SHARED-MEMORY ROUTER 



(57) Abstract 

The invention provides a single-chip method. The 
method includes a memory shared among packet buffers 
for receiving packets, packet buffers for transmitting 
packets, and packet header buffers for packet forwarding 
lookup. Accesses to that shared memory are multiplexed 
and prioritized. Packet reception is performed with 
relatively high priority, packet transmission is performed 
with medium priority, and packet forwarding lookup is 
performed with relatively low priority. The single-chip 
method includes circuits for serially receiving packet 
header information, converting that information into a 
parallel format for transmission to an SRAM for lookup, 
and queuing input packets for later forwarding at an output 
port. Similarly, the single-chip method includes circuits 
for queuing output packets for transmission at an output 
port, receiving packet forwarding information from the 
SRAM in a parallel format, and converting packet header 
information from output packets into a serial format for 
transmission. The single-chip method also includes a 
region in its shared memory for a packet forwarding table, 
and circuits for performing forwarding lookup responsive 
to packet header information. 
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Title of the Invention 

Single-Chip Architecture For Shared-Memory Router 

This application claims priority of Provisional Application No. 60/060,628, 
hereby incorporated by reference as if fully set forth herein. 

Background of the Invention 

1 . Field of the Invention 

This invention relates to a single-chip architecture for a shared-memory 



router. 
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2. Related Art 

In a packet-switched network, a "router" is a device which receives pack- 
ets on one or more input interfaces and which outputs those packets on one of a plural- 
ity of output interfaces, so as to move those packets within the network from a source 
device to a destination device. Each packet includes header information which indi- 
cates the destination device (and other information), and the router includes routing in- 
formation which associates an output interface with information about the destination 
device (possibly with other information). The router can also perform other operations 
on packets, such as rewriting the packets' headers according to their routing protocol or 
to reencapsulate the packets from a first routing protocol to a second routing protocol. 

It is advantageous for routers to operate, as quickly as possible, so that as 
many packets as possible can be switched in a unit time. Because routers are nearly 
ubiquitous in packet-switched networks, it is also advantageous for routers to occupy as 
little space as possible and to be easily integrated into a networking system. For exam- 
ple, implementing a router on a single chip (that is, a single integrated circuit) would be 
particularly advantageous. 

In this regard, one problem which has arisen in the art is that individual in- 
tegrated circuits and their packages are relatively limited in resources needed to imple- 
ment a router. In particular, individual chips have only a relatively limited number of 
pins, a relatively limited die area, and a relatively limited amount of power available for 
operation. These limitations severely limit the possibility of providing a useful router on 
a single chip. Routing devices generally need relatively more input and output ports 
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(thus requiring relatively more pins), relatively more lookup table space (thus requiring 
relatively larger die size for memory), relatively more packet buffering space (thus re- 
quiring relatively larger die size for memory), and relatively more packets routed in unit 
(thus requiring relatively larger die size for processing ability and relatively larger 
5 power dissipation for speed). 
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Accordingly, it would be advantageous to provide a single-chip router. 
This advantage is achieved in an embodiment of the invention in which a router inte- 
grated on a single chip shares memory among packet buffers for receiving packets, 
packet buffers for transmitting packets, and packet header buffers for packet forwarding 
lookup, and in which accesses to that shared memory are multiplexed and prioritized to 
maximize throughput and minimize routing latency. 

Summary of the Invention 



The invention provides a single-chip router. The router includes a memory 

1 7 shared among packet buffers for receiving packets, packet buffers for transmitting pack- 

18 ets, and packet header buffers for packet forwarding lookup. Accesses to that shared 

19 memory are multiplexed and prioritized. Packet reception is performed with relatively 

20 high priority, packet transmission is performed with medium priority, and packet for- 

2 1 warding lookup is performed with relatively low priority. 

22 

23 in a preferred embodiment, the single-chip router includes circuits for seri- 

24 ally receiving packet header information, converting that information into a parallel for- 

25 mat for transmission to an SRAM for lookup, and queuing input packets for later for- 
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, warding at an output port. Similarly, in a preferred embodiment,, the single-chip router 
includes circuits for queuing output packets for transmission at an output port, receiving 
packet forwarding information from the SRAM in a parallel format, and converting 
packet header information from output packets into a serial format for transmission. The 
single-chip router also includes a region in its shared memory for a packet forwarding 
table, and circuits for performing forwarding lookup responsive to packet header infor- 
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Brief Description of the Drawings 



Figure 1 shows a block diagram of a system including a single-chip router. 



Figure 2 shows a process flow diagram of a method for operating a system 
1 4 including a single-chip router. 

15 
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Detailed Description of the Preferred Embodiment 



In the following description, a preferred embodiment of the invention is 
described with regard to preferred process steps and data structures. Those skilled in the 

20 art would recognize after perusal of this application that embodiments of the invention 

21 can be implemented using circuits adapted to particular process steps and data structures 

22 described herein, and that implementation of the process steps and data structures de- 

23 scribed herein would not require undue experimentation or further invention. 
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System Elements 

Figure 1 shows a block diagram of a system including a single-chip router. 

A system 100 includes a single-chip router 110, a memory 120 coupled to 
the router 110 using a memory bus 121, and a processor 130 coupled to the router 110 
using a processor bus 131. 

The router 110 includes a plurality of input ports 111, a plurality of output 
ports 112, a memory port 113 coupled to the memory bus 121, a processor port 114 cou- 
pled to the processor bus 131, and a set of internal memory and internal processing cir- 
cuits integrated into a single monolithic integrated circuit on at least one side of a silicon 
die. 

In a preferred embodiment, the memory 120 includes an SRAM, and the 
memory bus 130 includes a 256 bit wide bus operating at about 125 megahertz, so as to 
provide 32 gigabits per second full duplex communication (that is, both to and from the 
memory 120). The memory 120 includes sufficient storage to record a set of packets 140 
which are received from the input ports 111 and which are in transit to the output ports 
112. 

The memory port 1 13 includes a memory data register 151 having 64 eight- 
bit bytes disposed in a set of four groups of 16 eight-bit bytes, and disposed for receiv- 
ing data from selected registers of the memory 120 (such as in memory read operations) 
and for transmitting data to selected registers of the memory 120 (such as in memory 
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write operations). The memory port 1 13 also includes a memory address register 152 for 
selecting the registers of the memory 120 to be read or written. 

Packet Receive Circuits 

Each one of the input ports 1 1 1 is coupled to an input MAC circuit 161 , for 
receiving a set of packets 140 from the input port 111, recognizing a MAC address of the 
sending device, recognizing a MAC address of the router 110 (as the receiving device), 
and coupling the packets 140 to an input packet queue 162. In a preferred embodiment, 
the input MAC circuit 161 receives the packets 140 in a bit serial format and outputs 
them to the input packet queue 162 as a sequence of eight-bit bytes. 

The input packet queue 162 includes a shift register, for receiving the se- 
quence of eight-bit bytes in serial from the input MAC circuit 161, and for transmitting a 
set of 256 bits (that is, 64 eight-bit bytes) in parallel to the memory data port 151. In a 
preferred embodiment, the input packet queue 162 is double-buffered; that is, it includes 
two separate shift registers, one of which can be reading packets 140 in serial from the 
input MAC circuit 161 while the other can be writing packets 140 in parallel to the 
memory data port 1 5 1 . 

The input packet queue 162 is coupled to a receive request circuit 163, for 
determining that the packet 140 has been received (or partially received, if more than 
256 bits in length), and for signalling the memory 120 to read the packet 140 from the 
input packet queue 162. The receive request circuit 163 is coupled to the memory ad- 
dress register 152 and to a control signal for the memory 120. 
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Packet Transmit Circuits 

Similar to the input ports 111, each one of the output ports 112 is coupled 
to an output MAC circuit 171, for transmitting a set of packets 140 from the output port 
112, adding a MAC address for the router 110 (as the sending device), adding a MAC 
address for the receiving device, and coupling the packets 140 from an output packet 
queue 172. In a preferred embodiment, the output MAC circuit 161 receives the packets 
140 as a sequence of eight-bit bytes and outputs them from the output packet queue 
172 in a bit serial format. 

Similar to the input packet queue 162, the output packet queue 172 in- 
cludes a shift register, for receiving a set of 256 bits (that is, 64 eight-bit bytes) in parallel 
from the memory data port 151, and for transmitting a sequence of eight-bit bytes in se- 
rial to the output MAC circuit 17 1 . In a preferred embodiment, the output packet queue 
172 is double-buffered; that is, it includes two separate shift registers, one of which can 
be reading packets 140 in parallel from the memory data port 151 while the other can be 
writing packets 140 in serial to the output MAC circuit 171. 

Similar to the input request circuit 163, the output packet queue 172 is 
coupled to a transmit request circuit 173, for determining that the packet 140- is ready to 
be transmitted (or partially ready, if more than 256 bits in length), and for signaling the 
memory 120 to write the packet 140 to the output packet queue 162. The transmit re- 
quest circuit 173 is coupled to the memory address register 152 and to a control signal 
for the memory 120. 
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Packet Address Lookup Circuits 

The input packet queue 162 is also coupled to a packet header queue 182, 
for isolating a packet header 141 for the packet 140 and for performing address lookup 
for that packet header 141. In a preferred embodiment, the packet header queue 182 re- 
ceives the packet header 141 in parallel from the input packet queue 162. 

The packet header queue 182 includes a shift register, for receiving a set of 
256 bits (that is, 64 eight-bit bytes) in parallel from the input packet queue 162, and for 
coupling the packet header 141 to an address request circuit 183. In a preferred em- 
bodiment, the packet header queue 182 is double-buffered; that is, it includes two sepa- 
rate shift registers, one of which can be reading packet headers 141 in parallel from the 
input packet queue 162 while the other can be coupling packet headers 141 to the ad- 
dress request circuit 183. 

the address request circuit 183 includes a hash circuit 184 for determining 
a hash address for packet lookup in the memory 120. The hash circuit 184 is coupled to 
the memory address register 152 for supplying a hash address to the memory 120 for per- 
forming packet lookup. The address request circuit 183 is also coupled to a control sig- 
nal for the memory 120. 



In a preferred embodiment, the hash circuit 184 is responsive to a (source, 
destination) pair in the packet header 141, such as described in detail in the following 
co-pending patent applications: 
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U.S. Application Serial No. 08/581,134, titled "Method For Traffic Management, 
Traffic Prioritization, Access Control, and Packet Forwarding in a Datagram Com- 
puter Network", filed December 29, 1995, in the name of inventors David R. 
Cheriton and Andreas V. Bechtolsheim, assigned to Cisco Technology, Inc., at- 
torney docket number CIS-019; 

U.S. Application Serial No. 08/655,429, titled "Network Flow Switching and 
Flow Data Export", filed May 28, 1996, in the name of inventors Darren Kerr and 
Barry Bruins, and assigned to Cisco Technology, Inc., attorney docket number 
CIS-016; and 

U.S. Application Serial No. 08/771,438, titled "Network Flow Switching and 
Flow Data Export", filed December 20, 1996, in the name of inventors Darren 
Kerr and Barry Bruins, assigned to Cisco Technology, Inc., attorney docket num- 
ber CIS-017. 



These patent applications are collectively referred to herein as the 
"Netflow Switching Disclosures". Each of these applications is hereby incorporated by 
reference as if fully set forth herein. 

The memory 120 responds to the hash address by delivering a set of 
packet lookup information to the memory data register 151, which is coupled to the 
packet header queue 182. The address request circuit 183 also includes a. comparator 
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185 for determining which of several packet lookup responses coupled to the packet 

header queue 182 is associated with the actual packet header 141. 

The packet header queue 182 is also coupled to the processor bus 131, for 
coupling packet headers 141 and packet lookup information to the processor 130 for 
extraordinary processing. Thus, when the router 110 is unable to process the packet 
140, or processing the packet 140 requires more flexibility than available to the router 
110 and the memory 120, the packet header 141 is coupled to the processor 130 for ex- 
traordinary processing. 

In a preferred embodiment, such extraordinary processing can include en- 
hanced packet forwarding and traffic management services such as access control, mul- 
ticast packet processing, random early discard, and other known packet processing 
services. 

System Operation 

Figure 2 shows a process flow diagram of a method for operating a system 
including a single-chip router. 

A method 200 includes a set of flow points to be noted, and steps to be 
executed, cooperatively by the system 100, including the router 110, the memory 120, 
and the processor 130. 
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At a flow point 210, an incoming packet 140 is received at one of the input 

ports 111. 

At a step 221, the input MAC circuit 161 receives the packet 140 and both 
recognizes the MAC address for the sending device, and confirms that the MAC address 
for the receiving device is the router 1 10. 

At a step 222, the input packet queue 162 receives the packet 140. 

At a step 223, the receive request circuit 163 determines a location in the 
memory 120 for the packet 140, and signals the memory 120 to receive the packet 140. 

At a step 224, the packet 140 is read into the shared memory 120 from the 
input packet queue 162. 

At a flow point 230, the packet 140 is ready to be routed. 

At a step 24 1 , the packet header 1 41 for the packet 140 is coupled from the 
input packet queue 162 to the packet header queue 182. 



At a step 242, the hash circuit 184 determines a hash address for the 
(source, destination) pair in the packet header 141, as described in the Netflow Switch- 
ing Disclosures, hereby incorporated by reference. 
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At a step 243, the address request circuit 163 couples the packet header 
15 i to the memory data register 151, couples the hash address to the memory address 
register 152, and signals the memory 120 to perform a packet address lookup. 

At a step 244, the memory 120 performs the packet address lookup and 
returns its packet lookup results to the memory data register 151. In a preferred em- 
bodiment, the memory 120 is disposed as a four- way set-associative memory responsive 
to the hash address provided by the hash circuit 184, so there are four packet lookup 
results. 

At a step 245, the comparator 185 determines which one of the four packet 
lookup results is valid for the (source, destination) pair in the packet header 141, and se- 
lects that one of the four packet lookup results for packet forwarding. 

At a flow point 250, the packet 140 is ready to be transmitted in response 
to the packet lookup results. 

At a step 261, the transmit request circuit 173 determines the location in 
the memory 120 for the packet 140, and signals the memory 120 to transmit the packet 
140. 

At a step 262, the packet 140 is read from the shared memory 120 into the 
transmit packet queue 172. 
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At a step 263, the output MAC circuit 171 both recognizes the MAC ad- 
dress for the sending device, and adds the MAC address for the receiving device (the 
router 110 itself), and transmits the outgoing packet 140 on the output port 112. 

At a flow point 270, an outgoing packet has been transmitted at one of the 
output ports 112. 

The router 110 operates with regard to each packet 140 using a parallel 
pipeline. Thus, a first packet 140 is being received while a second packet is being trans- 
mitted while a third packet 140 is having a packet lookup performed. 

The memory 120 has two regions (a packet buffer region for incoming and 
outgoing packets 140, and a packet header region for packet header lookup), each of 
which is intended to be accessed rapidly and often. However, multiple accesses to the 
memory 120 do not occur simultaneously; instead they are multiplexed so that accesses 
to these regions, are each serviced often by the memory 120, and prioritized so that ac- 
cesses to these regions can each be serviced rapidly by the router 110. 

In a preferred embodiment, packet reception is performed with relatively 
high priority, packet transmission is performed with medium priority, and packet for- 
warding lookup is performed with relatively low priority. 

Access requests by the receive request circuit 163 have the highest prior- 
ity, so that when requests for such accesses are received by the memory 120, they are 
processed before requests for accesses by other circuits. Thus, incoming packets are en- 
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1 tered into and retrieved from the input packet queue 162 as quickly as possible, so that 
queuing at the input ports 1 1 1 of the router 1 1 0 is minimized. 



Access requests by the transmit request circuit 173 have medium priority 
(after requests by the receive request circuit 163 and before requests by the address re- 
quest circuit 183), so that when requests for such accesses are received by the memory 
120, they are processed after requests for accesses by the receive request circuit 163 and 

8 before requests by the address request circuit 183. Thus, outgoing packets are entered 

9 into and retrieved from the output packet queue 172 as quickly as possible after incom- 
1 o ing packets are processed. 



11 

12 



Access requests by the address request circuit 183 have the lowest priority, 

1 3 so that when requests for such accesses are received by the memory 120, they are proc- 

1 4 essed after requests for access by other circuits. 



15 

16 Alternative Embodiments 



17 
18 



Although preferred embodiments are disclosed herein, many variations are 

1 9 possible which remain within the concept, scope, and spirit of the invention, and these 

20 variations would become clear to those skilled in the art after perusal of this application. 
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Claims 



2 

3 1. A router, including 

4 at least one port disposed for receiving packets; 

5 at least one port disposed for transmitting packets; and 

6 processing circuits integrated into a single monolithic integrated circuit on 

7 at least one side of a silicon die, said processing circuits including 

8 means for accessing a shared memory, said means including (a) circuits dis- 

9 posed for copying packets between at least one region of said shared memory and said 

I o processing circuits, and (b) circuits disposed for performing packet lookup in at least one 

I I region of said shared memory, said packet lookup being responsive to packet headers of 
12 said packets. 



13 

14 



2. A router as in claim 1, wherein said at least one port disposed for re- 

1 5 ceiving packets includes a plurality of ports disposed for receiving packets. 

16 

17 3. A router as in claim 1, wherein said at least one port disposed for 

1 8 transmitting packets includes a plurality of ports disposed for transmitting packets. 

19 

20 4. A router as in claim 1, wherein said circuits disposed for copying 

21 packets include circuits for receiving packet information in a parallel format and con- 

22 verting that information into a serial format. 

23 
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5. A router as in claim 1, wherein said circuits disposed for copying 
packets include circuits for receiving packet information in a serial format and convert- 



3 ing that information into a parallel format, 

4 
5 



6. A router as in claim 1, wherein said means for accessing said shared 
memory includes circuits disposed for multiplexing and prioritizing requests for copying 

7 incoming packets to said shared memory, requests for copying outgoing packets from 

8 said shared memory, and requests for performing packet lookup. 



7. A router as in claim 6, wherein said circuits disposed for multiplex- 

1 1 ing and prioritizing assign relatively high priority to said requests for copying incoming 

1 2 packets to said shared memory. 



8. A router as in claim 6, wherein said circuits disposed for multiplex- 

1 5 ing and prioritizing assign relatively low priority to said requests for copying outgoing 

1 6 packets from said shared memory. 



9. A router as in claim 6, wherein said circuits disposed for multiplex- 

19 ing and prioritizing assign priority to said requests in a manner so as to maximize 

20 throughput and minimize routing latency. 

21 

22 10. Apparatus including 

23 a shared memory, said shared memory including packet buffers for packets 

24 and packet lookup information; 
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! a router coupled to said shared memory, said router including processing 

i 

2 circuits integrated into a single monolithic integrated circuit on at least one side of a sili- 

3 con die, said processing circuits including 

4 means for accessing said shared memory, said means including (a) circuits 
disposed for copying packets between at least one region of said shared memory and 
said processing circuits, and (b) circuits disposed for accessing said shared memory for 
performing packet lookup. 



5 
6 
7 
8 
9 

1 o router. 
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21 



11. Apparatus as in claim 10, including a processor coupled to said 



12. Apparatus as in claim 10, wherein said means for accessing said 
shared memory includes circuits disposed for multiplexing and prioritizing requests for 

14 copying incoming packets to said shared memory, requests for copying outgoing pack- 

1 5 ets from said shared memory, and requests for performing packet lookup. 



13. Apparatus as in claim 12, wherein said circuits disposed for multi- 

18 plexing and prioritizing assign relatively high priority to said requests for copying in- 

1 9 coming packets to said shared memory. 



14. Apparatus as in claim 12, wherein said circuits disposed for multi- 

22 plexing and prioritizing assign relatively low priority to said requests for copying out- 

23 going packets from said shared memory. 
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15. Apparatus as in claim 12, wherein said circuits disposed for multi- 
plexing and prioritizing assign priority to said requests in a manner so as to maximize 
throughput and minimize routing latency. 

16. Apparatus as in claim 12, wherein said circuits disposed for copying 
packets include circuits for receiving packet information in a parallel format and con- 
verting that information into a serial format. 

17. Apparatus as in claim 12, wherein said circuits disposed for copying 
packets include circuits for receiving packet information in a serial format and convert- 
ing that information into a parallel format. 

18. A method, including the steps of 
receiving packets using at least one input port; 
transmitting packets using at least one output port; and 

routing said packets from said at least one input port to said at least one 
output port, using processing circuits integrated into a single monolithic integrated cir- 
cuit on at least one side of a silicon die. 



19. A method, including the steps of 
receiving packets using at least one input port; 
transmitting packets using at least one output port; 

copying packets between at least one region of a shared memory and a set 
of processing circuits integrated into a single monolithic integrated circuit on at least one 
side of a silicon die; and 
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perforaiing packet lookup using said shared memory and said processing 

circuits. 

20. A method as in claim 19, wherein said at least one input port in- 
cludes a plurality of ports disposed for receiving packets. 

21. A method as in claim 19, wherein said at least one output port in- 
cludes a plurality of ports disposed for transmitting packets. 

22. A method as in claim 19, wherein said step of copying packets in- 
cludes the steps of receiving packet information in a parallel format and converting that 
information into a serial format. 

23. A method as in claim 19, wherein said step of copying packets in- 
cludes the steps of receiving packet information in a serial format and converting that 
information into a parallel format. 

24. A method as in claim 19, wherein said steps of copying packets and 
performing packet lookup include the steps of multiplexing and prioritizing requests for 
copying incoming packets to said shared memory, requests for copying outgoing pack- 
ets from said shared memory, and requests for performing packet lookup. 



" 25 . A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign relatively high priority to said requests for copying incoming packets to 
said shared memory. 
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26. A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign relatively low priority to said requests for copying outgoing packets 
from said shared memory. 

27. A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign priority to said requests in a manner so as to maximize throughput and 
minimize routing latency. 

28. A method, including the steps of 

recording packets and packet lookup information in a shared memory; 

coupling processing circuits to said shared memory, said processing circuits 
being integrated into a single monolithic integrated circuit on at least one side of a sili- 
con die, said processing circuits including (a) circuits disposed for copying packets be- 
tween at least one region of said shared memory and said processing circuits, and (b) cir- 
cuits disposed for accessing said shared memory for performing packet lookup. 

29. A method, including the steps of 

recording packets and packet lookup information in a shared memory; 

copying packets between at least one region of said shared memory, and 
accessing said shared memory to perform packet lookup, using processing circuits which 
are integrated into a single monolithic integrated circuit on at least one side of a silicon 
die. 
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! 30. A method as in claim 29, including coupling a processor to said 

2 processing circuits. 



4 31 . A method as in claim 29, wherein said steps of copying packets and 

5 accessing said shared memory include the steps of multiplexing and prioritizing requests 

6 for copying incoming packets to said shared memory, requests for copying outgoing 
packets from said shared memory, and requests for performing packet lookup. 



9 32. A.method as in claim 31, wherein said steps of multiplexing and pri- 

10 oritizing assign relatively high priority to said requests for copying incoming packets to 

11 said shared memory. 



33. A method as in claim 31, wherein said steps of multiplexing and pri- 
oritizing assign relatively low priority -to said requests for copying outgoing packets 
1 5 from said shared memory. 

16 

! 7 34. A method as in claim 3 1 , wherein said steps of multiplexing and pri- 

18 oritizing assign priority to said requests in a manner so as to maximize throughput and 

1 9 minimize routing latency. 
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35. A method as in claim 31, wherein said steps of copying- packets in- 
clude the steps of receiving packet information in a parallel format and converting that 
23 information into a serial format. 

24 



WO 99/17182 PCTAJS98/20627 




36. A method as in claim 31, wherein said steps of copying packets in 
elude the sieps of receiving packet information in a serial format and converting that in 
formation into a parallel format. 
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